SysOp's Arsenal

home *** CD-ROM | disk | FTP | other *** search

/ SysOp's Arsenal / SysOp's Arsenal 1 (Arsenal Computer).ISO / netmail / imlfa20.exe / IMLFA20.DOC < prev next >

Wrap

Text File | 1994-04-20 | 30.7 KB | 647 lines

-= IMLFA =- Ver 2.0 InterMail Log file Analyzer/Reporter Copyright(c) 1993-1994 Gordon E. Rasnick, Arctic Computer Services 2530 Sebring Circle Anchorage, AK 99516 All rights reserved. FIDONET (1:3550/559) CompuServe ID: 72007,417 ******************* InterMail is Copyright 1988-1994 Scandinavian PC Systems AB & InterZone Software, Inc. All rights reserved. ******************* IMLFA version 2.0 Arctic Computer Services Page 1 ------------------------------------------------------------------------------ IMLFA is a complete log file analyzer/reporter for versions 2.2x of the InterMail E-mail package. IMLFA has only been tested with these releases, but may work with previous versions. No testing was done with earlier versions and no compatibility is assumed or implied. Care has been taken to attempt to eliminate any conflicts with future versions of InterMail log files, should formats change or additional information appear. Please read the documentation carefully. It is important that both IMLFA and InterMail be set up correctly for this system to work properly. This version of IMLFA is not crippled in any way. All the features are enabled in this release. When you first start it up, you will be asked to type in a number to continue. It will then perform without any further intervention. This random number step will occur each time you run IMLFA until you register your copy and receive a registration key code. See REGISTER.DOC for more information. Your registration will disable the random number step, permitting you to run it from your batch file during regular nightly maintenance events. SysOps/IM users.... You may distribute this software via any means you desire so long as it is distributed in the original self-extracting archive format it was released in, with all files intact and unmodified. You may not charge for this distribution with the exception of the cost of a distribution disk, if required, plus a minor shipping and duplication fee. Distribution by commercial corporations, companies, governmental agencies or educational institutions is prohibited without the expressed written consent of the copyright holder. This is usually not a problem, but solely for informational purposes. Page 2 Arctic Computer Services IMLFA version 2.0 ------------------------------------------------------------------------------ ******************* IMLFA requires at least 256k of free memory to run and will use any free EMS it finds. The more free conventional memory (0 - 640k) it has, the faster it will run. During beta testing, IMLFA processed InterMail log files of over 1 meg in about 3 minutes on a 386/40mhz machine with no EMS active. The 1+ meg log file contained transfer information on over 3000 file transfers with a busy HUB. Since IM's log file contains so much extra information, finding info about transfers between this HUB and a particular node would be a hair pulling experience. IMLFA transformed this 1 meg log file into a data file 10% of that size, still retaining all the transfer information such as, date, filename, file size, start time, stop time, average CPS throughput and connecting node. IMLFA will even flag you in its reports if a failed file transfer occurred and record the connect speeds of all your transfers. Once this data is imported into IMLFA, reports can be produced to provide you with information on your system's connects. The comments in the IMLFA.CFG file explain in detail the types of reports available. NOTES: IMLFA must be run from its own directory. It does not search paths for the IMLFA.* files. Change to the IMLFA directory before running this program. IMLFA does not modify InterMail's log file in any way. It does however, expect the InterMail log file to contain certain information. This may mean that you will have to modify your setup in IMSETUP to accommodate this program. This is explained in detail in the IMLFA.CFG file. If you find that you aren't getting what you expected out of this program, look in the IMLFA directory for a file called IMLFA.ERR. It will be created if trouble arises and provide you with information as to what the problem was. IMLFA version 2.0 Arctic Computer Services Page 3 ------------------------------------------------------------------------------ ******************* GETTING STARTED. 1. Create a directory on your hard disk called IMLFA. MD\IMLFA 2. Copy all the files from this archive to that directory. COPY *.* \IMLFA 3. Edit the IMLFA.CFG file to set up the system for the type of reports you wish to produce, the path and filenames. IMLFA.CFG is heavily commented and also serves as additional documentation for the program. Read it carefully and verify that all the paths you specify actually exist and are spelled correctly. You may now run IMLFA by passing any of the following command line parameters. NOTE: STACKING COMMAND LINE PARAMETERS IS NOT PERMITTED. o IMLFA -? produces a short screen of information on all of the available command line switches. o IMLFA -I processes all the data from the current IM log file. This obviously should only be done once per day and care should be taken to ensure that the log file is deleted, renamed or moved out of the way so that the data in it will not be imported again. (See the -B switch below.) o IMLFA -1:234/567=JOHN Will produce a detailed report of the traffic between you and John at node 1:234/567. Two reports will be produced, JOHN.IN and JOHN.OUT which include all INbound and OUTbound transfer information. You may use any legal file and path name you wish after the "=" sign, but USE NO EXTENSION. IMLFA -1:234/567=C:\BBS\TXTFILES\JOHN would create the report in the directory where you keep your bbs displayable text files. (C:\BBS\TXTFILES\) o IMLFA -C1:234/567=JOHN creates a report in the same format as above, but includes a Billing Total for the node based on either Total Bytes transferred or Total Connect time. See the "Billing" section below. o IMLFA -R will produce a single report of all the traffic between you and all other nodes you have had contact with. You specify the filename of this report in the IMLFA.CFG file at the MAIL FILE statement. The filename will always be the same and paths are supported here also. Note that the report is cumulative and shows Page 4 Arctic Computer Services IMLFA version 2.0 ------------------------------------------------------------------------------ all transfer information with your system since the IMLFA data files were created or last purged using the -W switch. o IMLFA -E permits you to edit any errors that may have occurred during the log file import. On rare occasions, after running IMLFA -R, you may notice a node entry (usually found at the very bottom of the report) which is listed as "UNKNOWN". This occurs when IMLFA just can't figure out what is going on with the log file. You can print a detailed report of this entry (IMLFA -UNKNOWN=<filename>) and then find the section of the IM log file containing the correct data. Use no extension for this filename. UNKNOWN.IN and UNKNOWN.OUT will be created as required. By calling IMLFA -E, you will enter an interactive mode that will allow you to correct the problem. Note that the -E switch only works when an entry for UNKNOWN exists and you may in fact never have a problem with the imports and not require this option. Online help is available during the edit and no command line parameters are required other than the -E switch. Should you have an import that produces an UNKNOWN entry, please forward a copy of the IM log file that created the problem to technical support at the node number listed in the SUPPORT section below. Without these log files, we can't find the problems. o IMLFA -M produces a single page summary of miscellaneous system information. There are many useful items such as, File Transfer totals, System Errors, Outbound call attempts, Inbound call attempts, Incoming BBS calls, Connect speeds, etc.... It may surprise you when you see just how many times your modem went off hook in a single week. NOTE: If you wish to make full use of this report, you should turn on DEBUG mode in the IMSETUP logging options. o IMLFA -B will back up the IM log file to the path specified in the BACKFILE statement in the IMLFA.CFG file. This option is very handy in several ways. First, after calling IMLFA -I to process your IM log file, you need to make sure that the log file is no longer available to be processed again. Using the -B switch will take care of that for you. Secondly, it will give you a means to recover your log file should you need to. IMLFA uses a date naming convention you can understand. Assume today's date is 01/02/93, it is just after midnight and you drop out of the mailer to process the log file. After running IMLFA with the -I switch, you run it again with -B, which renames the IM.LOG file to 01-01-93.LOG and moves to the directory you specified in IMLFA.CFG. Once all your processing is completed and your batch file restarts IM, a new log file is created. IMLFA version 2.0 Arctic Computer Services Page 5 ------------------------------------------------------------------------------ o IMLFA -T permits you to insert comments into your IM log file. Assume you have an external event set up to process inbound TIC's. IM's log file would probably contain an entry similar to: - 4:14:01 Executing event # 4, errorlevel: 220 Not very descriptive. By placing the following at the proper location in your batch file, things would be a bit more informative. REM Change to the IMLFA dir CD\IMLFA REM add comment to log file IMLFA -TProcessing_TIC_Files_From_1:234/567 REM Change back to the original directory CD\IM IM's log file would now show: - 4:14:01 Executing event # 4, errorlevel: 220 _ 4:14:02 Processing_TIC_Files_From_1:234/567 The maximum length of the text string is 60 characters. Should you attempt to write more than that, it will be truncated. NOTE: OS/2 users have the advantage of being able to pass a a string containing spaces, as long as it is enclosed in quotes. IMLFA -T"Processing TIC Files From 1:234/567" will work just fine in an OS/2 DOS box. o IMLFA -N creates all new index files for the stored data. If you see data that doesn't look quite right, you could have a corrupt index. Run IMLFA with this switch and then try running your reports again to see if the problem has been corrected. o IMLFA -ZA may be useful for HUB's wishing to use the reports for assisting in spreading out echo mail distribution costs. Since IMLFA stores data on all files transferred through your system, the data regarding files other than echo mail, FREQ'ed files, is included in all reports. Calling the -ZA switch will remove all files, containing the following extensions, from the stored data. .Z??, .A??, .PAK, .LZH, .EXE, .TXT, .TIC, .FLE, .PKT Use the -ZA switch prior to running your reports and you will receive a truer picture of just how much echo mail distribution your system is producing with each of your nodes. Page 6 Arctic Computer Services IMLFA version 2.0 ------------------------------------------------------------------------------ You may also pass a file extension if you wish to delete a file that wouldn't be included with the above. (-ZA.GIF will delete all occurrences of *.GIF files or -ZA.DOC will delete all *.DOC files from stored data.) o IMLFA -ZM will do the reverse of the above -ZA switch. FILENET HUBS may find this useful for creating reports for their downlinks regarding the filenet distribution. Calling this switch will delete files with the following extensions: .SUx .MOx .TUx .WEx .THx .FRx .SAx These are the standard echomail distribution extensions and consideration has been given to the instances where the 'x' would be higher than 9. (A-Z is used in this event.) o IMLFA -W will wipe out ALL stored data in the system, in effect, giving you a clean slate. CAREFUL! There is no escape from this switch. Since you want to be able to run it from your batch file, NO prompt will be given before the data is removed from the file, and that data is gone forever. Summary: You will probably want to call IMLFA several times in a session, first to process and back up the latest InterMail log, then to create the reports. A typical session in your batch file might look something like: :IM_RUN REM Change to the IMLFA directory. Remember....It doesn't search REM your paths for its files. CD\IMLFA REM Read in the latest IM log info. IMLFA -I REM Backup the log file you just processed. IMLFA -B REM I'm a Mail HUB so I want to Zap out all files not related to REM Echo Mail traffic. IMLFA -ZA REM Produce a detailed report of traffic to and from 1:234/567 IMLFA -1:234/567=C:\BBS\TXTFILES\JOHN IMLFA version 2.0 Arctic Computer Services Page 7 ------------------------------------------------------------------------------ REM Produce a detailed report of traffic to and from 1:234/568 IMLFA -1:234/568=C:\BBS\TXTFILES\JOE REM DITTO IMLFA -1:234/569=C:\BBS\TXTFILES\JIM REM Produce a summary report of all traffic with all nodes. REM This file is named and stored according to the information REM you provided in the IMLFA.CFG file. IMLFA -R REM Produce a miscellaneous report of all system activity. REM This file is also named and stored according to the information REM you provided in the IMLFA.CFG file. IMLFA -M REM Wipe out all data being stored by IMLFA. REM Use with caution as this is irreversible! REM Recommended that you do this in manual mode after you are REM certain that you have all the reports you wish from the REM stored data. IMLFA -W REM Start up BBS/IM again. GOTO RECYCLE The entire run above, on a months worth of IM log data, would only take a few minutes on a slow machine but you'll probably want to run this on a weekly basis or more often if you have a busy system. Mine runs every night, but without the call to IMLFA -W. I have that set up as a separate event occurring once a week and I have sysop only menus to read them all from a BBS logon. Run on a nightly basis, all of the above requires only a few seconds of downtime. REGISTERED USERS: It is recommended that you create an external event to exit at just after midnight to run the program, especially if you intend to use the log file backup option.(-B switch.) This would create a more realistic backup based on the date used as the filename. Page 8 Arctic Computer Services IMLFA version 2.0 ------------------------------------------------------------------------------ ******************* BILLING IMLFA provides two ways to assist in spreading the cost of echo mail distribution. Since the data provided by the IM log file is limited, so is this program, but with some cooperation from your downlinks, you should find it more than adequate. Cost may be distributed on a Total Connect Time Per Node or Bytes Transferred Per Node basis and is configured by the options you define for the BILLING command statement in the IMLFA.CFG file. BILLING=MINS,4 BILLING=BYTES,4 BILLING=N (default) (Remember, you also have the -ZA or -ZM switches for use in removing unwanted transfer information from the system. They give a truer picture of transfers for the type of distribution you do.) My NET uses an averaged Connect Time method which was agreed upon by all our nodes. Data for the cost of importing the echo mail was collected for several previous months and then averaged to a cost per minute among all our nodes. Yes, some nodes end up paying a bit more for less, but we're talking peanuts here, only pennies a day. It's simple, easily verified, and now, automated. (By the way, our number came to about 6.5 cents per minute to each node. Our net is small and long distance charges from Alaska are somewhat higher than the continental U.S., but with most of the nodes having 14400 modems, they can bring in 10 megs of compressed mail for about $6.00 a month.) Cost distributed by Bytes is probably the method to use if your downlinks use modems of varying speeds or range widely in the amount of mail they receive. It is still an averaged system, but spreads cost more realistically under the above conditions. The cost per bytes is actually a UNIT which IMLFA defines as 86400 bytes, or about 1 minute of transfer time at 14400 bps and at 100% throughput. Regardless of what speed a downlink connects at, his billing reflects each 86400 byte UNIT and if you have defined that unit to be charged at 5 cents, that is the amount his cost share will indicate regardless of his connect time. If he receives 172800 bytes, his cost is 10 cents regardless of the length of the connect. Your task in putting this to work for you is to decide just how much to charge, regardless of the method you choose. IMLFA version 2.0 Arctic Computer Services Page 9 ------------------------------------------------------------------------------ The Connect Time method is pretty straight forward. (cost per hour / 60) = cents per minute If the number is $3.50 per hour, 5.8 cents per minute is the parameter you use in the IMLFA.CFG file. (BILLING=MINS,5.8) You're downlinks would be charged 5.8 cents for every minute of connect time to you. The Bytes Transferred method requires a bit more math. (your cost / (total bytes from hub / 86400)) / number of downlinks. If you received 175,000,000 bytes and divided by 86400, you would have approximately 2025 UNITS. Divide that into your phone charges to your HUB, say $205.00, and you have roughly 10.1 cents per unit (your cost). (You'll probably notice that if you run a 14400 modem, the cents per unit will just about equal your telephone company's rate per minute for the call.) Divide this by your number of downlinks, .101 / 12 results in $0.008 cents. (BILLING=BYTES,.8) You can now see why the UNIT was established. On a true BYTE level, you would run out the decimal places beyond what most calculators want to deal with. You may wish to apply other variables to either of the equations. These figures aren't cast in stone, but are probably the best that can be done given the information the IM Log file provides. (The only truly accurate method involves using packet level software.) Yes, using the BYTES method, you may actually send out more than you receive and end up billing a bit more than you spend, but over a period of many months, it will probably average out. Otherwise, you can adjust the UNIT cost. The same holds true for MINS. In our net, all the nodes agreed that if the HUB collects too much over the months, everyone would get a freebie or the extra funds could be used for a Christmas party. The idea is to sell the system to your downlinks, then let the software do the work. Should anyone have an idea for another method, we'd love to hear about it. Keep in mind, your IM log file only tracks TIME and BYTES accurately. Page 10 Arctic Computer Services IMLFA version 2.0 ------------------------------------------------------------------------------ ******************** AUTO MODE Also new to IMLFA version 2.0 is the ability to set things up for a completely automatic system. Two options are available in the IMLFA.CFG file, AUTO REPORTS and AUTO RESET. Setting either or both of these options to Yes will cause IMLFA to watch for the first day of the month and then act upon the options you require. Auto reports set to YES will cause the summary and miscellaneous reports to be generated immediately following the import of the previous days log file. This would have the same result as passing the -R and -M switches from the command line or your batch file. Auto reset will remove all previously stored data from the system just as if you had passed the -W switch. Again, these options only occur when the date is the first day of a new month. IMLFA version 2.0 Arctic Computer Services Page 11 ------------------------------------------------------------------------------ ******************** MULTI-NODE CONSIDERATIONS If you are running a multi-node InterMail system, there are some things you may wish to change in your setup to make the most of this program. 1. It is recommended that you create copies of the IMLFA.CFG file equal to the number of nodes you run, naming them IMLFA1.CFG, IMLFA2.CFG, etc., and have your batch file COPY the appropriate file to IMLFA.CFG when you exit from each node in your events. Since each IM node will produce a separate log file, you will need to point each copy of the .CFG file to the proper IM log. (Version 2.1 of IMLFA will support the passing of .CFG file names on the command line.) 2. If you intend to use the IM log file backup option in IMLFA (called via the -B switch) you will need to ensure that each IM log file has a unique extension in its filename. IE: LOG.IM1, LOG.IM2... You will also want to create forced events for each of the nodes to exit with the appropriate errorlevel as soon as possible AFTER midnight. This gives your backups greater accuracy since they are based on the system dates. (See the -B switch description in the Available Switches section above.) 3. If more than one node tries to run IMLFA at the same time, the node making the second attempt will be put in a holding mode, and wait up to 5 minutes for access to the IMLFA data files. If access to the data files cannot be made, it will be recorded in the IMLFA.ERR file and IMLFA will return to the batch file without doing any further processing. Gaining access to the IM log file is handled in the same manner. To avoid this type of confusion, you could set the exits from IM to occur 2 or 3 minutes apart, but as with all forced events, they only happen when the IM node is in a wait state. If a transfer is in progress into and beyond the event, the possibility of a clash is quite real. The 5 minute wait should be sufficient for even the slowest systems capable of running multinode. 4. Unless you are running multiple copies (and storing multiple IMLFA data files for each node), it is strongly recommended that you permit only the last exiting node to use AUTO MODE, create daily reports or purge data files. This will help ensure that all IM log files have been processed. (creative use of semaphore files here could also prove useful.) Page 12 Arctic Computer Services IMLFA version 2.0 ------------------------------------------------------------------------------ 5. It has been reported that utilizing the FREENODE utility to gain access to and process all log files from a single exit will work, but it has not been extensively tested. If you attempt to process in this fashion, be sure to have some method in place to remove all processed IM log files before IMLFA runs again. (The -B switch sounds like just the ticket for this.) IMLFA version 2.0 Arctic Computer Services Page 13 ------------------------------------------------------------------------------ ******************** ERROR CHECKING IMLFA does a minimal amount of error checking as it performs its duties. Sometimes it will complain, sometimes it won't. This is primarily a batch file utility and assumes you have things set up properly. Once it is in place, things shouldn't change very often. Some of the things it DOES NOT check for: 1. Paths (directories\sub-directories) for the output of reports. Scanning a 1 or 2 gig disk for correct paths every time it is started up is a senseless waste of time. Be sure the paths you define actually exist and are spelled correctly since no attempt to create one will be made if it doesn't exist and no report will be written. 2. The existence of previously run reports. They are simply overwritten. 3. The possibility that the IM log file you are about to import has already been previously scanned. If you run IMLFA -I more than once on an IM log file, you'll end up with duplicate information in the database with no means of purging the dupes. Some of the things it DOES check for: 1. All the files it needs to operate properly. With the exception of the IM log file, all other system files must be in the current directory and IMLFA must be executed from that directory. If it doesn't find the files it needs, it will either try to create them, or, in the case of IM's log file, write an error message to the IMLFA.ERR file. 2. An existing IM log file Backup created by using the -B switch. You can only use that switch once per day and the IM log file is deleted when you do. 3. Valid node numbers passed as parameters at the command line. There are no errorlevels returned to DOS at any time. A future version may include this, but full blown error checking would need to be implemented for it to be effective. Page 14 Arctic Computer Services IMLFA version 2.0 ------------------------------------------------------------------------------ ******************** IMLFA SUPPORT Support is provided via NetMail or written request only. Responses via NetMail are the fastest, usually within 48 hours of receipt here. If there is a problem with log file imports, please bundle up a copy of the full day's IM log file, your IMLFA.CFG file and a copy of the report showing the problem, and send them to 1:3550/559. You do not need to send any of the IMLFA data files or other reports. They can be recreated here. I will also try to answer questions via my CompuServe address (listed above) if you so choose, but responses may not be as timely. Please DO NOT use the FidoNet InterMail echo for this purpose. I will not respond to questions or bug reports in that forum. I do this for 2 reasons. One, echos are cluttered enough without addressing the problems of shareware authors. Two, NetMail responses are MUCH faster. I will provide any help I can at FidoNet 1:3550/559 for as long as I remain in the FidoNet organization. I anticipate that being quite a while. :-) Comments and or suggestions for new features are welcome at any time. See REGISTER.DOC, included in this archive, for information on obtaining future fixes and/or upgrades. Enjoy..... Gordon